Business intelligence is a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making for an organization. Better decision-making is the driver for business intelligence. Generally, the process of providing business intelligence data starts with the determination of what kinds of summaries and reports a user may be interested in. Key business users are queried to determine the types of reports and summaries that they may be interested in. Due to the amount of resources required to effect changes in the types of reports and summaries that are generated, significant care is taken in designing these reports and summaries.
Once the required reports and summaries have been identified, the data required to generate these summaries and reports is determined. The data is typically stored by one or more Enterprise Information Systems (“EISes”), such as an Enterprise Resource Planning (“ERP”) system. These EISes are referred to herein as “source systems”. The particular location of the data in the source systems is noted, and extraction functions are coded to extract the data from the specific locations. In general, the goal of the extraction phase is to convert the data into a single format that is appropriate for transformation processing. Thus, the extraction functions not only retrieve the data from the source systems, but they parse and align the data with other data from the same or other source systems. As extraction functions have to be manually coded and tested for the data from each specific location, this step can be lengthy.
Transformation functions are then designed to transform and structure the data extracted from the source system(s) to enable rapid generation of the desired summaries and reports. The transform stage applies a series of rules or functions to the data extracted from the source system(s) to derive the data for loading into the end target. Some data sources require very little or even no manipulation of data. In other cases, one or more transformations may be required to be applied to the extracted data to meet the business and technical needs of a target database that is used to generate reports and summaries. Depending on the amount of transforming, the design and testing of the transformation functions can be a lengthy procedure.
Transformed data is then loaded into an end target, typically a data warehouse, that can be queried by users via business intelligence clients. Depending on the requirements of the organization, this process varies widely. Some data warehouses may overwrite existing information with cumulative information, frequently updating extract data is done on daily, weekly or monthly basis. Other data warehouses (or even other parts of the same data warehouse) may add new data in a historicized form, for example, hourly.
This process of providing business intelligence data is very manually intensive and requires significant expertise. The entire process typically takes from two to six months. Further, changes to the structure and/or format of the data in the source systems to be extracted can require significant manual recoding of the extraction functions. Further, changes to the information desired from the summaries and reports can require significant recoding of the extraction, transformation and load functions. As this is generally performed manually, the effort required can be substantial and is very sensitive to human error.
Accordingly, it is an object of the invention to provide a novel method and system for providing business intelligence data.